Procedural Modeling and Physically Based Rendering for Synthetic Data Generation in Automotive Applications

We present an overview and evaluation of a new, systematic approach for generation of highly realistic, annotated synthetic data for training of deep neural networks in computer vision tasks. The main contribution is a procedural world modeling approach enabling high variability coupled with physically accurate image synthesis, and is a departure from the hand-modeled virtual worlds and approximate image synthesis methods used in real-time applications. The benefits of our approach include flexible, physically accurate and scalable image synthesis, implicit wide coverage of classes and features, and complete data introspection for annotations, which all contribute to quality and cost efficiency. To evaluate our approach and the efficacy of the resulting data, we use semantic segmentation for autonomous vehicles and robotic navigation as the main application, and we train multiple deep learning architectures using synthetic data with and without fine tuning on organic (i.e. real-world) data. The evaluation shows that our approach improves the neural network's performance and that even modest implementation efforts produce state-of-the-art results.Comment: The project web page at http://vcl.itn.liu.se/publications/2017/TKWU17/ contains a version of the paper with high-resolution images as well as additional materia

A radiative transfer framework for non-exponential media

We develop a new theory of volumetric light transport for media with non-exponential free-flight distributions. Recent insights from atmospheric sciences and neutron transport demonstrate that such distributions arise in the presence of correlated scatterers, which are naturally produced by processes such as cloud condensation and fractal-pattern formation. Our theory accommodates correlations by disentangling the concepts of the free-flight distribution and transmittance, which are equivalent when scatterers are statistically independent, but become distinct when correlations are present. Our theory results in a generalized path integral which allows us to handle non-exponential media using the full range of Monte Carlo rendering algorithms while enriching the range of achievable appearance. We propose parametric models for controlling the statistical correlations by leveraging work on stochastic processes, and we develop a method to combine such unresolved correlations (and the resulting non-exponential free-flight behavior) with explicitly modeled macroscopic heterogeneity. This provides a powerful authoring approach where artists can freely design the shape of the attenuation profile separately from the macroscopic heterogeneous density, while our theory provides a physically consistent interpretation in terms of a path space integral. We address important considerations for graphics including energy conservation, reciprocity, and bidirectional rendering algorithms, all in the presence of surfaces and correlated media

Fluid Simulation for Visual Effects

This thesis describes a system for dealing with free surface fluid simulations, and the components needed in order to construct such a system. It builds upon recent research, but in a computer graphics context the amount of available literature is limited and difficult to implement. Because of this, the text aims at providing a solid foundation of the mathematics needed, at explaining in greater detail the steps needed to solve the problem, and lastly at improving some aspects of the animation process as it has been described in earlier works. The aim of the system itself is to provide visually plausible renditions of animated fluids in three dimensions in a manner that allows it to be usable in a visual effects production context. The novel features described include a generalized interaction layer providing greater control to artists, a new way of dealing with moving objects that interact with the fluid and a method for adding source and drain capabilities

Production volume rendering: design and implementation

Due to limited publicly available software and lack of documentation, those involved with production volume rendering often have to start from scratch creating the necessary elements to make their system work. Production Volume Rendering: Design and Implementation provides the first full account of volume rendering techniques used for feature animation and visual effects production. It covers the theoretical underpinnings as well as the implementation of a working renderer. The book offers two paths toward understanding production volume rendering. It describes: Modern production volume rendering techniques in a generic context, explaining how the techniques fit together and how the modules are used to achieve real-world goals Implementation of the techniques, showing how to translate abstract concepts into concrete, working code and how the ideas work together to create a complete system As an introduction to the field and an overview of current techniques and algorithms, this book is a valuable source of information for programmers, technical directors, artists, and anyone else interested in how production volume rendering works

Direct Transmittance Estimation in Heterogeneous Participating Media Using Approximated Taylor Expansions

Evaluating the transmittance between two points along a ray is a key component in solving the light transport through heterogeneous participating media and entails computing an intractable exponential of the integrated medium's extinction coefficient. While algorithms for estimating this transmittance exist, there is a lack of theoretical knowledge about their behaviour, which also prevent new theoretically sound algorithms from being developed. For this purpose, we introduce a new class of unbiased transmittance estimators based on random sampling or truncation of a Taylor expansion of the exponential function. In contrast to classical tracking algorithms, these estimators are non-analogous to the physical light transport process and directly sample the underlying extinction function without performing incremental advancement. We present several versions of the new class of estimators, based on either importance sampling or Russian roulette to provide finite unbiased estimators of the infinite Taylor series expansion. We also show that the well known ratio tracking algorithm can be seen as a special case of the new class of estimators. Lastly, we conduct performance evaluations on both the central processing unit (CPU) and the graphics processing unit (GPU), and the results demonstrate that the new algorithms outperform traditional algorithms for heterogeneous mediums.Funding: Knut and Alice Wallenberg Foundation (KAW) [2013-0076]; SeRC (Swedish e-Science Research Center); Wallenberg AI, Autonomous Systems and Software Program (WASP); Swedish Foundation for Strategic Research (SSF) via the project ASSEMBLE [RIT15-0012]; ELLIIT environment for strategic research in Sweden</p